Label Semantic Knowledge Distillation for Unbiased Scene Graph Generation

The Scene Graph Generation (SGG) task aims to detect all the objects and their pairwise visual relationships in a given image. Although SGG has achieved remarkable progress over last few years, almost existing models follow same training paradigm: they treat both object predicate classification as single-label problem, ground-truths are one-hot target labels. However, this prevalent paradigm overlooked two characteristics of current datasets: 1) For positive samples, some specific subject-object instances may have multiple reasonable predicates. 2) negative there numerous missing annotations. Regardless characteristics, easy be confused make wrong predictions. To end, we propose novel model-agnostic Label Semantic Knowledge Distillation (LS-KD) for unbiased SGG. Specifically, LS-KD dynamically generates “soft" label each instance by fusing predicted Distribution (LSD) with its original label. LSD reflects correlations between categories. Meanwhile, different strategies predict LSD: iterative self-KD synchronous self-KD. Extensive ablations results on three tasks attested superiority generality our proposed LS-KD, which can consistently achieve decent trade-off performance